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Background: In this paper we call for studying school leadership and its relationship to 
instruction and learning through approaches that highlight the role of configurations of 
multiple organizational supports. A configuration-focused approach to studying leadership 
and other essential supports provides a valuable addition to existing tools in school organiza¬ 
tional analysis and is particularly useful in examining equifinality and causal asymmetry. 
Equifinality is the idea that more than one pathway can result in a desired outcome whereas 
causal asymmetry suggests that the set of conditions that lead to the presence of an outcome 
need not be the same as the conditions that lead to its absence. 

Focus of Study: This study uses a configurational approach to examine how school leadership 
and other organizational supports are related to an important aspect of instruction — stu¬ 
dents ’ classroom participation. 

Research Design: We apply fuzzy set qualitative comparative analysis (QCA) to administra¬ 
tive and survey data of high schools from a large urban school district to examine combina¬ 
tions of organizational supports that are associated with classroom participation. 

Conclusions: The study draws attention to the utility of applying configurational approaches 
to investigate the influence of complex combinations of organizational supports on school 
outcomes. We compare this approach to more traditional methods that focus on the effects of 
isolated factors, controlling for each other. Our results show that leadership is associated with 
students’ classroom participation via multiple configurations of organizational supports. 
These configurations are different from the set of organizational supports that are related to 
an absence of classroom participation. 
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“There’s no such thing as a high-performing school without a 
great principal. It is impossible. You simply can’t overstate their 
importance in driving student achievement, in attracting and re¬ 
taining great talent to the school.” (U.S. Secretary of Education 
Arne Duncan as cited in Connelly, 2010, p. 35) 

“We claimed, based on a preliminary review of research, that lead¬ 
ership is second only to classroom instruction as an influence on 
student learning. After six additional years of research, we are 
even more confident about this claim. To date we have not found 
a single case of a school improving its student achievement re¬ 
cord in the absence of talented leadership.” (Louis, Leithwood, 
Wahlstrom, & Anderson, 2010, p. 9) 

“The preponderance of evidence indicates that school principals 
contribute to school effectiveness and student achievement indi¬ 
rectly through actions they take to influence school and classroom 
conditions (Hallinger & Heck, 1996a, 1996b). The size of the ef¬ 
fects that principals indirecdy contribute towards student learn¬ 
ing, though statistically significant, is also quite small.” (Hallinger, 
2005, p. 229) 

These quotes highlight a much debated contradiction in education research 
on the influence of school leadership on teaching and learning. An intui¬ 
tive belief in the importance of leadership and a wealth of evidence from 
qualitative research suggest that school leadership has a substantial influ¬ 
ence on classroom instruction and student learning in schools (Leithwood, 
Louis, Anderson, & Wahlstrom, 2004; Louis, Leithwood, et al., 2010). Yet 
evidence from quantitative studies suggests quite the contrary—that leader¬ 
ship effects are small and also largely indirect, working through an array 
of mediating organizational processes (Hallinger & Heck, 1996a, 1998). In 
a review of both quantitative and qualitative studies, Louis, Leithwood, et 
al. (2010) suggested two reasons for differing conclusions on the impor¬ 
tance of leadership. First, qualitative case studies are often done in atypi¬ 
cal settings, perhaps where leadership is exceptional or where leadership 
is required most. This could be one reason why qualitative studies report 
substantial leadership effects in contrast to quantitative studies that seek to 
estimate average effects over multiple settings. Second, even though quan¬ 
titative studies suggest small and indirect effects of leadership, it explains 
25% of the total variation between schools in student achievement. In this 
regard, only classroom instruction is more important than the role of lead¬ 
ership in influencing student achievement (Louis, Leithwood, et al., 2010). 
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In this paper we contend that another reason for apparent contradic¬ 
tions in the importance of leadership and other school processes arises 
from limitations of quantitative methods typically employed in school or¬ 
ganizational analysis. For example, given that there is emerging consensus 
that school leaders indirectly influence teaching and learning via multiple 
mediating factors (Hallinger & Heck, 1996a, 1998), two different analyti¬ 
cal approaches can be taken to clarify the role of leadership and mediat¬ 
ing factors. The first approach would typically answer a question such as 
this: Of all the organizational processes that leaders work on, which of 
them has significant and meaningful effects on teaching and learning, 
while controlling for everything else? Linear regression-based methods 
are best suited for this approach but are of limited use to practitioners 
because organizational processes do not exist in isolation—they naturally 
impact each other. The second approach would seek to answer a different 
question: What are the combinations or configurations of organizational 
processes that leaders work on that lead to strong teaching and learning, 
and what configurations undermine teaching and learning? The second 
approach is perhaps far more useful for school practitioners and policy 
makers. Yet little attention has been devoted to examining configura¬ 
tions of organizational supports systematically, and traditional quantita¬ 
tive methods are not well suited to studying configurations (Ragin, 2008; 
Ragin & Fiss, 2008). 

Here, we call for advancing empirical research on school leadership 
and school organizational analysis more broadly by demonstrating the 
importance of examining configurations of supports that influence im¬ 
portant school outcomes. Such an approach adopts a holistic principle of 
research where the focus is on patterns related to outcomes rather than 
on individual variables in isolation (Delery & Doty, 1996; Fiss, 2007, 2011). 
While configuration-based approaches have recently gained increased at¬ 
tention in fields such as business and marketing management, industrial 
organization, and studies of welfare states (see e.g., Crawford, 2012; Fiss, 
2011; Kent & Argouslidis, 2005; Kvist, 2007; Ordanini & Maglio, 2009; 
Sager & Andereggen, 2012), they have not been widely applied in educa¬ 
tional leadership and school organizational research. 

Adopting a configuration-based approach contributes to the research 
on school leadership and school organization in several ways. First, it ex¬ 
amines combinations of mediating school organizational supports that 
are important for school effectiveness rather than isolating the impor¬ 
tance of individual factors. Second, it extends theories of equifinality to 
the study of school effectiveness. Equifinality suggests that “a system can 
reach the same final state from differing initial conditions and by a variety 
of paths” (Katz & Kahn, 1978, p. 30). Gresov and Drazin (1997) clarify that 


3 


Teachers College Record, 116, 080305 (2014) 


“equifinality occurs when, in a sample of organizations, different structur¬ 
al alternatives yield the same functional effect” (p. 408). The concept of 
equifinality is receiving considerable attention in organizational research 
because it provides a theoretical basis for why several different organiza¬ 
tional design choices can all lead to effective outcomes (Fiss, 2011; Gresov 
& Drazin, 1997). Third, a configurational approach advances the notion 
of causal asymmetry—that the set of factors associated with the presence 
of an outcome need not be the same as those associated with the absence 
of an outcome (Ragin, 2008). As we show here, these advances allow for a 
more complete understanding of the relationships between school leader¬ 
ship, school organizational supports, and instruction and learning. 

While many methods can be used to examine configurations among 
school organizational supports (e.g., cluster analysis, latent class analysis, 
and multidimensional scaling), in this paper we use fuzzy set qualitative 
comparative analysis (QCA; Ragin, 2008). This method uses set theory 
to uncover necessary and sufficient sets of conditions as they relate to an 
outcome (Ragin, 2000, 2008). The approach offers several advantages 
over correlation-based quantitative methods such as multiple regression 
and structural equation modeling (SEM); we describe these advantages 
in detail later in the paper. The empirical data for this study comes from 
the Chicago Public Schools, using administrative data and teacher sur¬ 
vey data from biannual surveys administered by the University of Chicago 
Consortium on Chicago School Research (CCSR). 

We examine the same data used in a recent study of school leadership 
(Sebastian & Allensworth, 2012) that examined the influence of school 
leadership on teaching and learning via an array of mediating organi¬ 
zational supports using multilevel structural equation modeling (SEM). 
Using the same data allows us to compare the findings from the pres¬ 
ent study (using a configuration-based approach) to the Sebastian and 
Allensworth (2012) study that uses a correlation-based approach. One dif¬ 
ference from that study is that we do not examine student achievement as 
an outcome but focus on classroom instruction. Adopting a multifaceted 
conceptualization of classroom instruction that focuses on interactions 
among teachers and students around educational material (Cohen & 
Ball, 1999), the CCSR teacher surveys collect information on multiple di¬ 
mensions of instruction (Bryk, Sebring, Allensworth, Luppescu, & Easton, 
2010). These include academic demands, critical thinking, quality of stu¬ 
dent discussions, classroom disorder, time on homework, and student 
participation (Bryk et al., 2010; Sebastian & Allensworth, 2012). In the 
current study, we limit the analysis to examining one aspect of classroom 
instruction—student participation, which measures the extent to which 
students perform tasks expected of them, come to class on time, attend 
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class regularly, turn in their assigned homework, come adequately pre¬ 
pared and pay attention in class, and actively participate in class. 1 While 
student participation is just one feature of the quality of classroom instruc¬ 
tional environments (Cohen & Ball, 1999), the Sebastian and Allensworth 
(2012) study found this aspect to be a particularly strong mediator of stu¬ 
dent achievement in high schools. It also closely corresponds to the ele¬ 
ment of instruction that the Bill and Melinda Gates (2011) Measures of 
Effective Teaching (MET) study found to be most strongly associated with 
student learning gains, classroom control, that was measured through sur¬ 
veys of middle grade students. 

Using fuzzy set QCA, we show that leadership is associated with stu¬ 
dent participation through different configurations of essential organi¬ 
zational supports. The set of organizational supports that are associated 
with high levels of student participation are quite different from the set 
of supports that are related to an absence of student participation, which 
are again different from supports associated with very high student par¬ 
ticipation. We conclude by discussing the implications of the study for 
school leadership and organizational research. In discussing the limita¬ 
tions of the study we also outline a series of next steps for empirical re¬ 
search in studying leadership and school organizational supports using 
a configuration-based approach. 

RESEARCH ON SCHOOL LEADERSHIP 

Most studies of school leadership, regardless of whether they use qualita¬ 
tive, quantitative, or mixed methods, typically acknowledge the complex¬ 
ity inherent in the work of school leaders. Myriad practices and multiple 
modes of school leadership have been associated with student achievement 
(Hallinger & Heck, 1996a, 1998; Marks & Printy, 2003; Marzano, Waters, 
& McNulty, 2005). One of the most salient roles of a school principal for 
improving student achievement is as an instructional leader. This includes 
a direct role through their expertise in content and pedagogy and indi¬ 
rectly through supporting good instructional practices (Hallinger, 2005; 
Hallinger, Bickman, & Davis, 1996; Hallinger & Heck, 1998; Halverson, 
Grigg, Prichett, & Thomas, 2007; Louis, Dretzke, & Wahlstrom, 2010; 
Stein & D’Amico, 2000; Stein & Nelson, 2003). To show improvements 
in achievement over time, they also need to be transformative leaders— 
building capacity, inspiring and motivating employees, and developing 
commitment to organizational goals (Bass, 1998; Leithwood & Jantzi, 
1999, 2005; Marks & Printy, 2003). Making the study of leadership more 
complex, there is increasing recognition that there are sources of leader¬ 
ship situated outside the principal and distributed among teachers and 
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other key school personnel (Camburn, Rowan, & Taylor, 2003; Spillane, 
2006; Spillane, Camburn, & Pareja, 2007). Thus, the conceptualization of 
what leadership is, and what leaders should do to improve instruction and 
student learning, is multidimensional. 

Besides being a complex role, it is difficult to measure leadership ef¬ 
fects on student achievement as these effects are largely indirect, work¬ 
ing via mediating processes such as school climate, culture, and capacity 
(Hallinger & Heck, 1996a, 1996b, 1998, 2010a, 2010b; Witziers, Bosker, & 
Kruger, 2003). Bryk et al. (2010), for example, contend that school lead¬ 
ership works via three key mediating processes to influence teaching and 
learning: the professional capacity of the staff, parent-community ties, and 
the school climate. 2 They combine to influence instruction in classrooms, 
which in turn influences student learning (see Figure 1). 


Figure 1. Conceptual model of school leadership, mediating processes, 
instruction, and student learning 



Thus, a key methodological challenge in studying the relationship be¬ 
tween school leadership and school outcomes is using appropriate meth¬ 
ods to clarify the role of organizational factors/supports that mediate the 
influence of leadership on teaching and learning. Structural equation 
modeling (SEM) has been useful in examining the strength of direct and 
indirect relationships of leadership with school outcomes via multiple me¬ 
diating factors (Hallinger & Heck, 1996a, 1996b, 2010a, 2010b; Supovitz, 
Sirinides, & May, 2010). However, most studies of leadership focus on a 
limited set of factors that are often unique to the study, making compari¬ 
sons across studies difficult (Louis, Dretzke, et al., 2010). A limited num¬ 
ber of studies have considered the influence of leadership via multiple 
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mediating processes simultaneously (see e.g., Hallinger et al., 1996a; 
Louis, Leithwood, et al., 2010). These studies use a net effects approach 
(Ragin, 2008; Ragin & Fiss, 2008) that treats variables as competing with 
each other for explaining variation in an outcome; the focus is to estimate 
the effects of individual variables in isolation, net of, or controlling for, 
other variables (Fiss, 2007, 2011; Ragin, 2000, 2008; Ragin & Fiss, 2008). 
For example, Sebastian and Allensworth (2012) found learning climate to 
be the only significant mediating factor “holding constant” other mediat¬ 
ing processes such as parent/community ties, professional development, 
professional community, and school contextual factors. 

Methods such as multiple regression and SEM are suitable for isolat¬ 
ing the relative importance of independent variables net of each other, 
but as Louis, Dretzke, et al. (2010) note, “most school variables consid¬ 
ered separately, have only small effects on student learning. To obtain 
large effects, educators need to create synergy across relevant variables” 
(p. 10). Leadership studies face methodological constraints in uncover¬ 
ing this synergy or in studying optimal combinations of relevant organiza¬ 
tional variables. Theoretically, the effects of combinations of variables or 
configurations can be studied in regression through interaction effects. 
However, modeling configurations through interaction effects in regres¬ 
sion can be challenging for at least three reasons. First, the number of 
interactions that can be included in a regression model is limited by the 
sample size, making it difficult to implement in studies that do not have a 
large sample of schools. Second, when interactions go beyond two-way or 
three-way interactions, the results become hard to interpret (Fiss, 2007). 
Last, when there are multiple mediating factors, it is difficult to determine 
which combinations and how many higher order interactions to include 
in the regression model. Because of these inherent methodological limita¬ 
tions, school leadership studies typically acknowledge the importance of 
configurations of organizational supports but do not systematically exam¬ 
ine relevant and optimal configurations. 

Another limitation of traditional quantitative methods such as multiple 
regression is that they often assume unifinality—which suggests one opti¬ 
mal configuration for all cases for achieving success in the outcome (Fiss, 
2007). This methodological constraint stands in stark contrast to leader¬ 
ship and school organizational theory that often cautions against a “one 
size fits all approach” (Louis, Leithwood, et al., 2010, p. 101) to school 
leadership effectiveness or a single panacea for organizing schools for ef¬ 
fective reform (Brooks, Scribner, & Eferakorho, 2004). Furthermore, the 
importance of context is invariably highlighted in most leadership and 
school organizational research. Yet, the importance of match between con¬ 
text and organizational strategy is not adequately addressed in empirical 
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research (Hallinger et al., 1996a). Lastly, organizational studies using 
quantitative methods often assume that if there is an association between 
the presence of a factor and an outcome, then absence of that factor must 
be associated with absence of the outcome. It is safe to say that concepts 
such as equihnality and causal asymmetry have not been empirically ex¬ 
amined in education leadership and school organization research, even 
though they offer great promise in explaining how different leadership 
styles and organizational strategies can all lead to school performance and 
how different sets of conditions can be important in determining success 
versus contributing to failure in outcomes. 

CONFIGURATIONAL APPROACHES TO STUDYING SCHOOL LEADERSHIP 
AND ORGANIZATION 

The present study proposes an alternate approach to studying leadership 
by examining configurations of essential school supports that leaders fo¬ 
cus their work on and that are associated with teaching and learning. A 
configurational approach acknowledges the context dependent and com¬ 
plex nature of organizations (Delery & Doty, 1996; Doty & Click, 1994) 
where “parts of a social entity take their meaning from the whole and can¬ 
not be understood in isolation” (Meyer, Tsui, & Hinings, 1993, p. 1178). 
This approach is more consistent with theory on school leadership and 
school organization that typically acknowledges that leaders have to struc¬ 
ture an array of organizational supports to influence the core activity of 
schools—classroom instruction. As Knapp, Copland, and Talbert (2003) 
note, in order to improve student learning, successful leaders act purpose¬ 
fully along multiple pathways that strategically “take advantage of events, 
relationships, conditions, and resources within a particular setting and 
time” (p. 23). The configurational approach proposed here can ultimately 
result in holistic theories of school leadership that connect leadership, 
school organizational structures, leadership strategies, and the school 
context to important outcomes such as student achievement. 

We use a set-theoretic method, qualitative comparative analysis (QCA) 
to study schools from a configurational approach; these methods are well 
suited for this purpose as they conceptualize observations as combinations 
of attributes as they relate to important outcomes (Fiss, 2007; Ragin, 2000, 
2008). The QCA method was developed by Ragin (1987) to analyze bi¬ 
nary data and later developed to work with nonbinary data as well (Ragin, 
2000, 2008; Rihoux & Ragin, 2009). Boolean algebra is used to examine 
which combinations of attributes result in an outcome (Fiss, 2011; Ragin, 
2000, 2008). The following section provides a brief overview of set-theo¬ 
retic methods. Readers are referred to literature specific to set-theoretic 
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methods (see e.g., Fiss, 2011; Ragin, 2000, 2008; Ragin & Fiss, 2008) for 
more detailed descriptions of these procedures. 

Consider a researcher with dichotomous (yes/no) data from a school 
district on several school organizational characteristics including “strong 
school leadership” and “strong professional community of teachers” and 
one school outcome “strong gains on student achievement.” Consider 
that the researcher also has information on school contextual character¬ 
istics such as whether the school has a large student enrollment (“school 
size”) and if the school is a selective enrollment school based on students’ 
prior achievement (“high prior achievement”). If the researcher finds two 
results: (a) all schools with strong gains on achievement also have strong 
leadership and (b) all schools with strong school professional community 
have strong gains in achievement, this illustrates two important conditions 
in set-theoretic methods (Ragin, 2000, 2008). The first result (Figure 2) 
depicts the necessary condition indicated when instances of an outcome 
are a subset of instances of a condition. A necessary condition must be 
present for the outcome to also be present, but it may not be enough 
on its own. The second result (Figure 3) depicts a sufficient condition 
indicated when instances of the condition are a subset of instances of the 
outcome. A sufficient condition produces the outcome but may be one of 
several conditions that does so. 


Figure 2. Set-subset diagram representing necessary condition 



Set of schools with 
strong leadership 


Set of schools with 
strong student 
achievement 
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These examples show how important connections can be missed by 
correlational analysis (Ragin, 2008). In the first result, the set-theoretic 
method establishes that school leadership is a necessary condition for 
achievement gains, as all schools with strong student achievement also 
have strong leadership. But the presence of schools with strong leader¬ 
ship that do not meet the outcome (strong gains in achievement) can re¬ 
sult in a weak correlation between leadership and achievement gains. The 
fact that there are many schools with strong leadership that do not have 
strong gains in student achievement does not undermine a claim that 
school leadership is a necessary condition; those schools may simply lack 
other conditions that need to accompany strong leadership for a school to 
show high achievement. The set-theoretic approach makes intuitive sense 
because it is likely the case that strong leadership alone cannot result in 
effective schools; rather leadership may need time to foster other factors 
such as teacher capacity and strong professional development in order for 
schools to be successful. 


Figure 3. Set-subset diagram representing sufficient condition 



In Figure 3, the fact that all schools with strong professional community 
have strong student achievement indicates that this condition is sufficient 
on its own to produce strong achievement gains. As with the leadership 
example in Figure 2, the fact that there are schools with strong gains in 
achievement that do not have strong professional community does not 
undermine the claim that a strong professional community of teachers is a 
sufficient condition for gains in achievement. There might be alternative 
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pathways through which schools might reach the same outcome even 
though those pathways might be more difficult for schools to follow. These 
pathways perhaps involve strengthening of other organizational structures 
such as professional development to compensate for weakness in teacher 
community. Perhaps contextual factors such as selective enrollment of 
high achieving students can play a role, so that a strong professional com¬ 
munity of teachers is not necessary if the school only serves highly moti¬ 
vated, academically strong students. Figure 4 depicts such a hypothetical 
situation where either strong professional community or selective enroll¬ 
ment of students with high prior achievement lead to achievement gains. 

Figure 4. Set-subset diagram representing multiple sufficient conditions 



In Boolean notation the Venn diagram in Figure 4 is expressed as: 
Professional Community + High Prior Achievement Achievement Gains 
read as: Professional Community or High prior achievement lead to stu¬ 
dent achievement gains. In Boolean notation, the + sign stands for the 
logical OR (union of conditions), stands for the logical AND (intersec¬ 
tion of conditions), and stands for the absence of a condition. Typically 
researchers are interested in complex combinations of organizational and 
contextual variables that lead to outcomes. QCA is well suited for this as 
it can analyze complex causation or association (Ragin, 2008), defined as 
“situations in which an outcome may follow from several different combi¬ 
nations of causal conditions, that is, from different causal recipes” (p. 23). 
In our previous example a hypothetical result could be: 

Strong School Leadership • Professional Community 
+ High Prior Achievement School Size Achievement Gains 
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This statement introduces a contingency hypothesis (Fiss, 2007), i.e., 
schools with strong leadership will have high student achievement gains, 
provided they also have a strong professional community. Similarly, 
schools enrolling students with high prior achievement will also produce 
achievement gains provided they maintain a small school size (absence of 
large school size). As noted by Fiss (2007), these Boolean statements can 
very elegantly summarize the match between organizational characteris¬ 
tics, context, and performance. 

Formal analysis of complex combinations of organizational structures 
and contextual features that result in an outcome starts in QCA with con¬ 
struction of the truth table—a list of every single combination of condi¬ 
tions along with the outcome associated with each combination (Ragin, 
2008; Ragin & Fiss, 2008). If there are k conditions in the analysis, this will 
result in a truth table with 2* rows. Each observation or case in the study is 
categorized into this truth table based on their values for the conditions. 
Therefore some rows on the truth table will have many cases, implying 
that there are many observations that have the particular combination of 
attributes; some rows will have few cases, and some none (Fiss, 2011; Ragin, 
2008). From the truth table, two important concepts in QCA are used to 
reduce the 2* rows in the table. Coverage refers to the minimum number of 
cases for a particular combination to be considered as a relevant solution 
by the researcher (Ragin & Fiss, 2008). When the sample size of a study is 
small, a researcher might want to consider any configuration of conditions 
that result in the outcome. For medium sized or larger samples, the re¬ 
searcher might be more restrictive, requiring at least two or three cases for 
a solution (i.e., a particular combination of conditions) to be considered. 
For medium or large sized samples, considering every combination that 
leads to an outcome might not be useful as some information might simply 
be noise and the solutions will be too numerous to interpret. Consistency 
refers to the extent to which cases have a common configuration of also 
display the outcome of interest (Ragin, 2000, 2006; Rihoux & Ragin, 2009). 
Thus, when considering a solution, if half the cases displaying that particu¬ 
lar combination show the outcome but the rest of the cases do not display 
the outcome, the consistency of that solution cannot be considered very 
high. Consistency can be estimated by the proportion of cases displaying a 
particular combination of conditions that also displays the outcome (Fiss, 
2007; Ragin, 2008). The minimum recommended consistency level is 0.75 
(Ragin, 2006, 2008) or higher (Fiss, 2011). 

Boolean algebra is used to simplify the results from the truth table that 
list the various configurations that are sufficient for the outcome. Consider 
for example the following hypothetical solutions for the study described 
earlier: 
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Strong School Leadership • Professional Community School Size 

Achievement 

Strong School Leadership • Professional Community • School Size 
-* Achievement 

Clearly both the presence or absence of large school size are associated 
with the outcome, and these two statements can be logically reduced to 
one Boolean statement: 

Strong School Leadership • Professional Community -* Achievement. 

Using similar operations to reduce the number of combinations, the 
Quine-McCluskey algorithm is used to obtain simple statements that con¬ 
tain all possible combinations that lead to the outcome (see Fiss, 2007, 
2011; Ragin, 2000, 2008). 

The procedures used to logically reduce configurations associated with 
an outcome also include a step termed counterfactual analysis (Fiss, 2007, 
2011; Ragin, 2000, 2008). As described earlier, if there are k conditions 
considered in a study the truth table will have 2* combinations. This leads 
to the problem of limited diversity, which means that often there will be 
a limited number or no instances of particular combinations that will 
complicate the reduction of combinations into simple statements using 
Boolean algebra. Counterfactual analysis is a method to deal with lack of 
empirical instances of particular combinations by including substantive 
and theoretical knowledge about conditions. The reader is referred to 
(Fiss, 2007, 2011; Ragin, 2000, 2008) for detailed descriptions of counter- 
factual analysis. Very briefly, there are two types of counterfactual analysis, 
“easy” and “difficult.” When a redundant condition is added to a configu¬ 
ration because theoretical knowledge suggests that the cause is associated 
with the outcome, it is called an easy counterfactual. In contrast, if a con¬ 
dition is assumed to be redundant and is removed from a configuration 
associated with the outcome, even though prior theory links the presence 
of the condition with the outcome, it is termed a difficult counterfactual 
(Fiss, 2011; Ragin, 2008). 3 

There are three types of solutions to QCA analysis depending on whether 
easy and/or difficult counterfactuals are included to simplify the solution. 
A parsimonious solution includes both easy and difficult counterfactuals 
to get the simplest solutions. An intermediate solution only includes easy 
counterfactuals. A complex solution does not include any counterfactual 
analysis. The intermediate solutions that consider only easy counterfactu¬ 
als are usually of most interest to researchers (Fiss, 2011). In short, when 
substantive knowledge and prior research is used to simplify the solution 
we obtain an intermediate solution. On the other hand, if all possible 
combinations of conditions are used to simplify the solution, regardless 
of prior research and substantive knowledge, we obtain the parsimonious 
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solution. Easy and difficult counterfactuals also determine core and pe¬ 
ripheral conditions. Core conditions show a strong association with the 
outcome and appear in the intermediate and parsimonious solutions; pe¬ 
ripheral conditions show weaker evidence of a relationship with the out¬ 
come and are only part of intermediate solutions (Fiss, 2011). 

Binary Data to Fuzzy Sets 

The QCA methods described so far are applicable to binary data. However, 
because most phenomena in social science do not have a binary character 
and vary by degree, set theoretic methods were not widely used until re¬ 
cently in social science research (Ragin, 2000; Ragin & Fiss, 2008). Recent 
mathematical developments allow the application of set theoretic methods 
to nonbinary data through the use of fuzzy sets. Fuzzy sets allow for mem¬ 
bership in a set to be less restrictive than binary sets, and instead of just two 
values zero or one, researchers can calibrate “partial membership in sets” 
with values that range in between zero and one (Ragin, 2000; Ragin & Fiss, 
2008). They can be seen as a continuous variable that has been converted 
to a fuzzy set using calibration (Ragin, 2000, 2008; Rihoux & Ragin, 2009). 
Researchers use thresholds that are based on substantive knowledge to de¬ 
fine set membership with values close to zero indicating nonmembership 
and values close to one denoting full membership in that set. 

Consider for example, a researcher examining the quality of princi¬ 
pal leadership in a school based on the proportion of teachers endors¬ 
ing their principal’s leadership. The researcher can aggregate informa¬ 
tion from teachers to the school level and convert this information to a 
fuzzy set of principal endorsement using at least three values, 0, 1, and 0.5. 
Nonmembership in the fuzzy set of principal endorsement is denoted by 
0, 1 denotes full membership and 0.5 denotes a crossover point, a point 
of maximum ambiguity or fuzziness (thus the name fuzzy sets) and where 
membership is neither in nor out (Ragin, 2008). Substantive knowledge 
has to be used in deciding where to set these thresholds. The researcher in 
this example might set the benchmark of 0.5 in the fuzzy set of principal 
endorsement for schools where 50% of teachers endorse the principal 
but 50% do not, clearly a situation where there is ambiguity about princi¬ 
pal endorsement. Creation of fuzzy sets usually involves more than three 
benchmarks to produce more refined fuzzy sets. 

With calibration, all the original raw scores are converted to range 
from zero to one based on anchors chosen as thresholds; an intermedi¬ 
ate step involves transforming scores into a log odds metric (Fiss, 2011). 
Calibration or the setting of meaningful benchmarks to create fuzzy sets 
is another instance that involves the researchers’ substantive knowledge 
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to proceed with QGA analysis. Recall that substantive knowledge is also 
required in considering easy and difficult counterfactuals to simplify the 
analysis of Boolean statements. The creation of fuzzy sets allows us to use 
the set-theoretic concepts and operations described earlier in relation to 
binary data, such as necessary and sufficient conditions, creation of truth 
tables, and logical reduction of combinations of conditions using Boolean 
algebra (Ragin, 2008). 


RESEARCH QUESTIONS 

In this study we examine the relationship of school leadership and the 
organizational supports of school climate, professional capacity of staff, 
and parent-community ties as they relate to school organizational per¬ 
formance. The configuration approach described here permits the study 
of different combinations of essential supports that are associated with 
school performance. The method also allows us to separately examine 
if these are the same supports (when absent, for example) that are as¬ 
sociated with a lack of school performance instead of assuming this to be 
the case. In order to make clearer comparisons to correlation-based ap¬ 
proaches such as regression and SEM, we use the same data and the same 
conceptual model previously used by Sebastian and Allensworth (2012) in 
the study of the mediating processes between leadership and classroom 
instruction. The key research question they focused on was: Which areas 
of school leaders’ work are most strongly related to classroom instruction 
and student learning? Here we instead focus on necessary and sufficient 
conditions of organizational supports including school leadership that are 
associated with school performance. Our specific research questions are: 

• What combinations of organizational supports are associated with 
school performance? Are these the same supports that are also as¬ 
sociated with absence of school performance? 

• Is school leadership necessary for school performance? And con¬ 
versely, is lack of school leadership necessary for absence of school 
performance? 

As described earlier, a difference from the Sebastian and Allensworth 
(2012) study is that we are focusing on one aspect of instruction as an in¬ 
dicator of school performance and not examining student achievement. 
The difficulties of using QCA to examine student achievement outcomes 
for the present data lies primarily in determining meaningful thresholds 
to create fuzzy sets of student outcomes at the school level (especially after 
accounting for students’ incoming achievement). 
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RESEARCH METHODS AND DATA 

The data used for this study come from high schools that are part of the 
Chicago Public Schools (CPS), the third largest school system in the 
United States. The student population is about 50% African American, 
38% Latino, 9% White, and 3% Asian. Approximately 85% of students 
are eligible for free/reduced priced lunches. About two thirds of Chicago 
high school students remain in school to graduate. This study used two 
data sources: administrative and test data from CPS and teacher sur¬ 
vey data. The teacher survey data come from the University of Chicago 
Consortium on Chicago School Research’s (CCSR) survey administered 
in the spring semester of the 2006-2007 academic year on a range of is¬ 
sues including classroom instruction, professional development, learn¬ 
ing climates, parent involvement, and principal leadership. The overall 


response rate for the 2007 teacher surveys was 71.6%. Data for a total of 
4,317 teachers from 98 high schools were used in this study. Table 1 com¬ 
pares the demographic and student performance characteristics for all 
CPS high schools and the high schools that participated in the survey and 
are included in the analysis. 

Table 1. Performance and Demographic Characteristics of Schools 


2006- 

-2007 

Characteristic 

Schools in 

All CPS high 


sample 

Schools 

(1 V- Schools) 

98 

no 

Average enrollment 

1013 

974 

Achievement: ACT composite score 

16.63 

16.80 

Racial composition (%) 



African American 

62.98 

60.35 

Latino 

26.80 

29.34 

White 

5.76 

5.80 

Truancy rate 

9.68 

9.63 

Graduation rate 

66.89 

68.25 

Attendance rate 

85.24 

85.76 

Students—% low income 

83.92 

83.45 

Mobility rate 

22.35 

21.71 
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ANALYTIC MODEL 

As described earlier, we use fuzzy set QCA to examine how configurations 
of organizational conditions are associated with a school outcome. The 
computer program fs/QCA (Ragin, Drass, & Davey, 2006) was used to con¬ 
duct the analysis. We set the consistency and coverage thresholds based on 
recommendations of prior research on using QCA methods (Fiss, 2011; 
Ragin, 2008). We set the lowest acceptable consistency for solutions to 
the minimum recommended value of 0.75 (Ragin, 2008). Coverage is like 
strength or importance; it indicates the relevance of a solution. QCA was 
initially developed as a small N method where every configuration related 
to meeting the outcome could be considered for solutions. More recendy, 
QCA has been extended to larger samples with larger coverage values re¬ 
quired. For example, Fiss (2011) used a coverage value of three cases for a 
sample of 205 technology firms. With our sample of 98 schools, we set the 
minimum acceptable coverage to two—at least two cases had to display a 
specific configuration of conditions related to the outcome to be consid¬ 
ered as a solution. 

DEPENDENT MEASURES 

In this study we focus our analysis on one aspect of instruction as the out¬ 
come—students’ classroom participation. The importance of classroom 
participation and student engagement for student learning has been ex¬ 
tensively documented in prior literature. Researchers contend that class¬ 
room participation is most proximally related to student learning (Finn, 
1993) and that it is important regardless of subject area (Newmann & 
Associates, 1996; Turner & Patrick, 2004). Prior research has shown that 
classroom participation is associated with student grades, attendance, aca¬ 
demic competence, and achievement (Finn, 1993; Finn & Voelkl, 1993; 
Ladd, Birch, & Buhs, 1999; Valiente, Lemery-Chalfant, Swanson, & Reiser, 
2008). The Sebastian and Allensworth (2012) study used a measure of 
classroom instruction that comprised of several dimensions including stu¬ 
dent participation, classroom disorder, quality of student discussions, and 
critical thinking; among these measures, student participation had the 
strongest factor loading on an underlying latent measure of instruction 
and also showed the most between-school variance among all dimensions 
included in instruction. 

Ladd et al. (1999) identify two types of classroom participation; at one 
level students engage in the tasks and demands of the class by showing up 
for class, being prepared with supplies, textbooks, following directions, 
and completing homework. At a higher level, they actively participate and 
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take initiative by asking questions and working independently (Ladd et ak, 
1999). The composite measure for student participation in this study con¬ 
tains both aspects. It is derived from questions asking teachers how many 
students in a target class: come to class on time, attend class regularly, 
come prepared with the appropriate supplies and books, regularly pay 
attention in class, actively participate in class activities, and always turn in 
their homework. Teachers were asked to provide information about their 
second period class on Mondays. A scale was created using Rasch analy¬ 
sis (Wright & Masters, 1982), producing a measure with high reliability 
(0.89). Because the surveys were administered in the spring, teachers had 
considerable experience with their target classes to draw some general 
and valid conclusions about them. 

The analysis with fuzzy set QCA requires transforming variables into 
calibrated sets in relation to three substantively meaningful thresholds: 
full membership, full nonmembership, and the crossover point. Based 
on these benchmarks a continuous variable is converted to scores denot¬ 
ing membership in a fuzzy set with transformations that are based on 
the log odds of membership (Ragin, 2000). However, much of research 
in social science does not use calibrated measures; benchmarking is not 
a commonly practiced procedure (Ragin, 2008). Therefore, there is no 
prior research to guide the benchmarking process necessary for con¬ 
ducting fuzzy set QCA for this data. The original response categories for 
questions about student participation were gradations of agreement: 1 = 
strongly disagree, 2 = disagree, 3 = agree, 4 = strongly agree. We created 
an average across all items and then rounded that average to a binomial 
value to indicate whether a teacher was more in agreement or disagree¬ 
ment on the set of items that were related to classroom participation of 
students. Then, for each school we estimated the proportion of teachers 
who were in agreement that students were participating in their class. In 
order to account for differences in survey response (some schools had 
better response rates from teachers), we weighted information using hi¬ 
erarchical linear models (HLM; Raudenbush & Bryk, 2002). A nonlinear 
HLM (see the appendix for details) was used to calculate a weighted 
estimate for each school of the proportion of teachers who agreed on 
questions about student participation in their classrooms. The high cor¬ 
relation (0.90) between this estimated value (proportion of teachers 
who agree that students participate in class) with the school average of 
the original Rasch measure (see Table 2), shows that little information is 
lost with the transformation. 
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Table 2. Correlations between Original Rasch Measures and Proportion 
of Endorsers 

Rasch measures 

Proportion endorsing (school level) 

Instructional leadership 

0.89 

Teacher influence 

0.95 

Collective responsibility 

0.96 

Teacher collaboration 

0.90 

Program coherence 

0.93 

Parent-teacher interaction 

0.89 

Teacher-parent trust 

0.90 

Teachers 

0.96 

College expectations 

0.92 

Student participation 

0.90 


We created two fuzzy set outcome measures based on student classroom 
participation. For the first fuzzy set, membership in the set of schools with 
high student participation was coded one if at least 75% of teachers in a 
school agreed that students actively participate in the classroom and was 
coded zero if 25% of teachers or less agreed that students participate in 
class. As the crossover point we chose 50% to indicate that if only half the 
teachers in school felt that students participated in classroom activities, 
then that denoted the point of maximum ambiguity or the fuzzy point. 
The second fuzzy set measure, membership in the set of schools with very 
high student participation, was coded one if 90% of the teachers or more 
agreed that students actively participate in class and zero if less than 50% 
of the teachers agreed that students participate in class. As the crossover 
point, we chose 75% of teachers agreeing on student participation. Low 
student participation was simply coded as the negation of high student 
participation described above (one for < 25% of teachers in agreement, 
zero for >75% agreement, and the crossover point at 50%). 

MEASURES OF SCHOOL LEADERSHIP AND SCHOOL ORGANIZATIONAL 
PROCESSES 

We use the Bryk et al. (2010) conceptual model (Figure 1) as the basis 
for the analysis plan. School leadership in the Bryk et al. (2010) model 
focuses on multiple leadership roles of the principal: developing the mis¬ 
sion and goals of the school, setting high standards and communicating 


19 




Teachers College Record, 116, 080305 (2014) 


expectations to staff, developing trust and collaboration among staff, and 
supporting instruction. It also includes a distributed leadership perspec¬ 
tive, defined as the influence of teachers in school decision-making pro¬ 
cesses. Although distributed leadership need not to be restricted to the 
roles of teachers alone (Spillane, 2006; Spillane et al., 2007), teacher lead¬ 
ership is typically emphasized, as teacher influence and distributed lead¬ 
ership are often used interchangeably (Robinson, Lloyd, & Rowe, 2008). 
Therefore, the conceptualization of distributed leadership in this particu¬ 
lar model primarily focuses on principals and teachers. 

The measures available to us are identical to those used in the Bryk et al. 
(2010) study. We use a measure of principal instructional leadership that re¬ 
flects the extent to which teachers see their principals as instructional leaders 
with respect to setting standards for teaching and learning, communication 
of a clear vision, and tracking academic progress. A measure of teacher influ¬ 
ence reflects the extent to which teachers are involved in various aspects of 
school decision making. The descriptions of the other mediating organiza¬ 
tional supports, their influence on classroom instruction and student learn¬ 
ing, and the importance of school contextual factors have been described 
in prior research (see e.g., Bryk et al., 2010; Sebastian & Allensworth, 2012). 
As with classroom participation, composite measures have been developed 
based on previous empirical work (Bryk et al., 2010), and Rasch analysis 
(Wright & Masters, 1982) was used to create the original measures. 

Principal instructional leadership (alpha = 0.91) is made from ques¬ 
tions about whether the principal sets high standards for teaching, knows 
what is going on in the classroom, has clear expectations for meeting in¬ 
structional goals, communicates a clear vision, understands how children 
learn, carefully tracks student academic progress, participates in instruc¬ 
tion planning, presses teachers to implement what they have learned in 
professional development, and has clear expectations for meeting instruc¬ 
tional goals. Teacher influence (alpha = .81) reflects the degree to which 
teachers are involved in school decision making. Questions ask of teach¬ 
ers’ influence on the selection of instructional materials, setting of school 
policy, in-service program planning, discretionary funds spending, setting 
standards for student behavior, and hiring of professional staff. The trans¬ 
formation of these variables to fuzzy scores replicated the same steps and 
same benchmarks used for transforming the measure of classroom partici¬ 
pation to a fuzzy set of high classroom participation. 

Professional capacity is described with two measures. Program coherence 
(alpha = .79), describes the quality and coordination of programs in the 
school that support instruction. Another measure, peer collaboration (al¬ 
pha = .73) captures one aspect of the professional community at the school 
level by describing how much teachers collaborate around instruction and 
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the extent to which they engage in deprivatized practice by observing other 
teachers or inviting other teachers to observe their own instruction. Two 
measures are used to capture the learning climate present in the school. 
Teachers’ perceptions of crime and disorder (alpha = .89) is highly corre¬ 
lated with students’ perceptions of school safety and captures problematic as¬ 
pects of the environment (Steinberg, Allensworth, & Johnson, 2011). School 
orientation toward postsecondary education or college expectations (alpha 
= .79) captures the academic environment—the degree to which the school 
is pressing students to have high academic achievement. Two measures cap¬ 
ture the quality of parental and community ties. Teacher-parent trust (alpha 
= .67) measures the extent to which parents and teachers support each other 
to improve student learning and feel mutual respect. Parent-teacher inter¬ 
actions (alpha = .58) measure the degree to which teachers contact parents 
when there is some academic problem with their children or when their 
children have performed well in class. For the specific items that comprise 
each measure the reader is referred to prior research using CGSR measures 
(Bryk et al., 2010; Sebastian & Allensworth, 2012). The transformation of 
these variables to fuzzy scores also replicated the steps taken for transform¬ 
ing the measures of instructional leadership, teacher influence, and other 
organizational supports to fuzzy set variables. Benchmarks of > 75%, = 50%, 
and < 25% were used to denote the points of full membership, the crossover 
point, and full nonmembership for these fuzzy sets. 

SCHOOL CONTEXTUAL VARIABLES 

The models also included important school level contextual variables— 
school size, average prior ability of the students in the school, and percent¬ 
age of low income students in the school (as measured by eligibility for free 
and reduced price lunch). For the fuzzy set of schools of large size, schools 
that had over 1,500 students were fully in, schools with 600 students were 
fully out, and as the crossover point we chose a school size of 900 students. 
These categories correspond to those used in studies on high school size 
(Lee, 2004; Lee & Smith, 1997) and also correspond closely to the 75th, 
50th, and 25th percentiles of school size for the sample of high schools used 
in the study. Lee and Smith (1997) labeled schools with 600-900 students 
as middle-sized high schools and found that learning gains were highest 
in these schools. Prior ability was measured using students’ scores on the 
EXPLORE standardized test on ACT’s Education Planning and Assessment 
System (EPAS). In 2007, all CPS ninth graders took the EXPLORE early 
in the fall and therefore this test represents a good measure of their prior 
achievement. In 2007, the national average on the EXPLORE was 15; using 
this score as a benchmark we calculated the percentage of students who 
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scored at least a 15 on the EXPLORE test. For the fuzzy set of schools with 
a high proportion of students coming adequately prepared academically 
(at least a 15 on the EXPLORE), schools where > 75% of students met this 
criteria were fully in, schools where <25% of students met the criteria were 
fully out, and the crossover point was kept at 50%. For incoming achieve¬ 
ment, a more readily available category was also used; whether the school 
was a selective enrollment school or not. The use of this variable is also 
useful to illustrate how crisp data (where variables take on binomial values, 
zero or one) can be used together with fuzzy set data (where variable have 
values ranging from zero to one) in QCA. 

For the socioeconomic status of the school, the proportion of students 
on free and reduced lunch was used. Specifically for the fuzzy set of schools 
with high poverty, schools where > 75% of students were eligible for free 
and reduced lunch were fully in, schools where < 25% of students met 
the criteria were fully out, and the crossover point was kept at 50%. These 
categories correspond to the categories used by the National Center for 
Education Statistics (NCES) in their Condition of Education series (see 
e.g., Aud et ah, 2011, 2012); these studies use consistent benchmarks for 
defining high-poverty and low-poverty schools based on student eligibility 
for free/reduced lunch to allow for comparability across years. These cat¬ 
egories might be considered too restrictive for the Chicago context where 
most schools serve high proportions of low-income students. On the other 
hand, they allow for comparability with studies such as the Condition of 
Education series that look at national samples. They also underscore the 
fact that most Chicago high schools serve low-income students; analyzing 
relative differences among these schools might lead to erroneous conclu¬ 
sions if the context is not emphasized. 

The above benchmarks for leadership, school organization, and contex¬ 
tual variables make intuitive sense, are not sample driven, and are meant 
to begin a discussion on benchmarking school organizational and contex¬ 
tual variables. The fuzzy sets of high classroom participation, high leader¬ 
ship, and high organizational supports have arguably lenient benchmarks 
of 50% teacher agreement for the fuzzy point and 25% agreement to be 
fully out. One could contend that if only half the teachers are in agree¬ 
ment about leadership, or classroom participation, that should qualify for 
being fully out in corresponding fuzzy sets rather than being the point of 
ambiguity. To consider this, we redid all analysis with more conservative 
benchmarks where benchmarks of > 75% was used to denote full mem¬ 
bership, < 50% was used to denote full nonmembership, and > 62% was 
used to denote the crossover point or the fuzzy point. These benchmarks 
correspond closely to those used by Fiss (2011) in a study of typologies of 
high technology firms using QCA. 
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We considered the intermediate solutions obtained from the QCA anal¬ 
ysis wherein substantive knowledge is used for conducting the counterfac- 
tual analysis. We assumed that the presence of all school organizational 
and contextual factors except for large school size and the proportion of 
students on free and reduced lunch were linked to higher student par¬ 
ticipation in classrooms. We also assumed that the absence of those two 
conditions, large school size and proportion of students on free and re¬ 
duced lunch, were linked to higher student participation in classrooms. 
The most parsimonious solutions that consider all possible combinations 
in order to simplify the solution regardless of substantive knowledge were 
not considered. Comparing solutions by relaxing or changing various as¬ 
sumptions were beyond the scope of the present study but would be valu¬ 
able to further school organization research using QCA methods. 

It could be argued that none of the benchmarks used thus far for any of 
the fuzzy sets are appropriate. The benchmarks for creating the fuzzy set 
of classroom participation may be too lenient, while the benchmarks for 
parent-teacher interactions may be too conservative; high schools in gen¬ 
eral may have a limited amount of parent teacher interaction in contrast 
to elementary schools, where such interaction might be more frequent. 
Also, one might argue that the benchmarks for creating fuzzy set measures 
of school poverty and incoming achievement are not useful to the Chicago 
context where most schools serve more than 75% low-income students, es¬ 
pecially if a researcher is not very concerned about comparisons to other 
contexts. Future studies could focus on determining meaningful and ap¬ 
propriate benchmarks that are more specific to local contexts. Lastly, rela¬ 
tive variation in a measure might be important in explaining differences 
between schools in terms of important outcomes. Correlation-based ap¬ 
proaches are well suited to examining relative variation among variables 
and have already been applied to this dataset before (see Sebastian & 
Allensworth, 2012). The set theoretic approach followed in this study is an 
additional tool for research on school organizations and offers a different 
approach by requiring researchers to make decisions related to relevant 
(rather than relative) variation based on substantive knowledge. 

RESULTS 

The descriptive statistics and correlations for all measures used in this study 
are presented in Table 3. The descriptive statistics for teacher survey mea¬ 
sures are not based on the fuzzy sets but from the HLM-based Empirical 
Bayes estimates that represent proportions of teachers who agree across 
items for each measure. The descriptive statistics show that on average, the 
proportion of teachers who agree that their principal is an instructional 
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leader is quite high (0.78). The average agreement across schools for teach¬ 
er safety (0.36) and parent-teacher interaction (0.31) is much lower. As 
expected, the correlation of classroom participation with teacher safety is 
high (0.72), and the correlation of classroom participation with college ex¬ 
pectations is moderate (0.62). Also as expected, there is a strong correlation 
of classroom participation with incoming achievement (0.72). 


Table 3. Descriptive Statistics and Correlations 



Variable 

Mean 

s.d. 

1 2 

3 

4 

5 

6 

7 

8 9 

to 

11 

1 

Enrollment 

(size) 

Proportion 

1013.76 

800.66 











students with 
free/reduced 

0.84 

0.15 

-0.15 









2 

lunch 

Proportion 

students 

with > 15 on 

0.35 

0.23 

0.25 -0.72 









3 

EXPLORE 












4 

Instructional 

leadership 

0.78 

0.11 

0.00 0.03 

-0.08 








5 

Teacher 

influence 

0.55 

0.16 

-0.31 -0.09 

0.13 

0.47 







6 

Teacher 

collaboration 

0.49 

0.08 

-0.04 0.07 

-0.20 

0.42 

0.30 






7 

Program 

coherence 

0.51 

0.12 

0.04 -0.23 

0.20 

0.59 

0.48 

0.40 





8 

Parent- 

teacher trust 

Parent- 

0.67 

0.11 

-0.05 -0.42 

0.45 

0.35 

0.39 

0.11 

0.54 





teacher 

0.31 

0.05 

0.04 0.08 

-0.13 

0.11 

0.00 

0.08 

0.16 

0.11 



9 

interaction 












10 

Teacher 

safety 

0.36 

0.29 

-0.20 -0.42 

0.61 

0.13 

0.40 

-0.04 

0.47 

0.58 -0.03 



11 

College 

expectations 

0.70 

0.19 

-0.19 -0.47 

0.61 

0.19 

0.42 

0.07 

0.40 

0.72 -0.04 

0.70 


12 

Classroom 

participation 

0.56 

0.16 

0.08 -0.49 

0.72 

-0.03 

0.25 

0.00 

0.33 

0.54 -0.14 

0.72 

0.62 


Table 4 shows the results for a fuzzy set QCA analysis for the outcome of 
high student participation. Here we use the notation developed by Ragin 
and Fiss (2008) by which dark circles (“•”) indicate the presence of a con¬ 
dition, and a circle with a cross (“m”) indicates the absence of a condition. 
Core conditions are indicated by large circles; these conditions are present 
in both the intermediate and parsimonious solutions and are strongly relat¬ 
ed to the outcome. Peripheral conditions are indicated by small circles; these 
conditions are present only in the intermediate solutions and have weaker 
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evidence of a relationship with the outcome. A blank space means that the 
condition can be absent or present and indicates a “don’t care situation” 
(Fiss, 2011). We ran two different sets of analysis, one with a fuzzy set of in¬ 
coming achievement based on proportion of students coming in with at I cast 
15 on the EXPLORE and the second with a dichotomous indicator of wheth¬ 
er the school was a selective enrollment school or not. The results from both 
sets of analysis for the outcomes of high student participation and very high 
student participation are virtually identical. The results discussed below are 
for models that used the fuzzy set based on average incoming achievement. 

Table 4. Configurations Related to High Classroom Participation 

Solution 

Configuration 1 2 

Context 

Large size 
High poverty 

High incoming achievement 

Leadership 

Principal instructional leadership • • 

Teacher influence • 

Organizational supports 

Peer collaboration • 

Program coherence • 

Teacher safety 

College expectations • 

Teacher-parent trust • 

Parent-teacher interaction 


Consistency 

0.71 4 

0.86 

Raw coverage 5 

0.89 

0.49 

Unique coverage 6 

0.41 

0.02 

Overall solution consistency 

0.71 


Overall solution coverage 

0.91 


• Core condition (presence) 

• Peripheral condition (presence) 



The solution table (Table 4) shows that the fuzzy set analysis for high 
classroom participation results in two solutions exhibiting the presence of 
both core and peripheral conditions. The results indicate two solutions for 
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high participation highlighting equihnality of solutions. The core condition 
for the first solution is high expectations of students whereas the core condi¬ 
tion for the second solution is high teacher influence in school policy. Both 
solutions indicate the presence of instructional leadership, indicating that 
instructional leadership from the principal seems to be a necessary condi¬ 
tion. At the same time, leadership is not enough. There are no schools that 
meet the outcome solely on the basis of strong principal or teacher leader¬ 
ship. Solution 1 indicates that instructional leadership combined with high 
college expectations can lead to high student participation in classrooms, 
regardless of the school context. The other solution to obtaining high stu¬ 
dent participation involves instructional leadership and teacher influence 
but requires many more of the essential supports. There is less coverage for 
this solution indicating that many more schools achieve success with the 
configuration involving high academic expectations. 

Table 5. Configurations Related to Very High Classroom Participation 


Solution 


Configuration 

i 

Context 

Large size 

High poverty 

High incoming achievement 

• 

Leadership 

Principal instructional leadership 

• 


Teacher influence 

• 

Organizational supports 

Peer collaboration 

Program coherence 

• 


Teacher safety 

• 


College expectations 

• 


Teacher-parent trust 

• 

Consistency 

Parent-teacher interaction 

0.85 

Raw coverage 


0.48 

Unique coverage 


0.48 

Overall solution consistency 


0.85 

Overall solution coverage 


0.48 

• Core condition (presence) 




• Peripheral condition (presence) 
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Table 5 shows the results of the QCA analysis for very high student partici¬ 
pation and here asymmetry is highlighted—the set of conditions that lead 
to an outcome may well be different from the set of conditions that lead 
to higher degrees of the outcome. As Table 5 shows, the core condition 
for very high student participation in classrooms is the presence of a high 
proportion of students that come in with adequate academic preparation 
(at least a 15 on the EXPLORE). 

However, not all schools that have a high proportion of students with 
adequate academic preparation show very high student participation. The 
presence of almost all essential supports, instructional leadership, teacher 
influence, program coherence, teacher safety, college expectations, and 
teacher-parent trust are required together with the core condition of high 
incoming student achievement. The number of schools that meet the out¬ 
come of very high classroom participation is only 15 schools to begin with. 
Therefore the solutions shown in Table 5 explain the patterns of very few 
schools indicating that it is difficult for schools to attain very high classroom 
participation and that there is little consistency in their patterns. 

Finally, Table 6 shows the results of the fuzzy set QCA analysis for absence 
of student participation in classrooms. The results indicate that absence of 
teacher safety is a core condition that leads to schools meeting this out¬ 
come. Lack of teacher safety as a core condition is combined with the pe¬ 
ripheral conditions of absence of a high proportion of students coming in 
with adequate academic preparation, the presence of a high proportion of 
students classified as low-income students, and the absence of parent-teach¬ 
er interaction. Notice that principals’ instructional leadership and teacher 
influence do not matter for this solution. This solution can be interpreted 
to indicate that lack of safety, regardless of principal or teacher leadership, 
is the critical condition that results in low classroom participation. 

Table 6. Configurations Related to Absence of Classroom Participation 

Solution 

Configuration 1 

Context 

Large size 

High poverty • 

High incoming achievement m 

Leadership 

Principal instructional leadership 

Teacher influence 
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Solution 

Configuration 1 

Organizational supports 

Peer collaboration 
Program coherence 

Teacher safety m 

College expectations 
Teacher-parent Trust 


Parent-teacher interaction m 

Consistency 0.65 

Raw coverage 0.92 

Unique coverage 0.92 

Overall solution consistency 0.65 

Overall solution coverage 0.92 

m Core condition (absence) 
m Peripheral condition (absence) 

• Peripheral condition (presence) 

DISCUSSION 


A configuration-focused approach shows that different conditions are 
required to achieve high student participation, when compared to the 
school conditions required for very high student participation in class¬ 
rooms, which are again different from the configurations that correspond 
to low student participation. It also shows that certain conditions are only 
a possibility—or a risk—in schools serving particular populations of stu¬ 
dents. These differences might not be discerned using a traditional regres¬ 
sion-based approach. 

For the outcome of absence of student participation, the most important 
condition is a lack of teacher safety in schools. Neither principal instruc¬ 
tional leadership nor distributed leadership among teachers is adequate 
to compensate for low school safety. In schools where teachers feel unsafe, 
students also report feeling unsafe (Steinberg et ah, 2011), and this fear 
may prevent students from engaging fully in classroom instruction and 
may also influence teachers’ capacity to engage students as they deal with 
concerns for their own safety. It also suggests that only certain types of 
schools are at risk for very low student participation—those serving many 
students in poverty and schools not serving high achieving students. 

School leadership seems to be a necessary condition to meet the 
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outcome of high student classroom participation. In order to have high 
student participation, where at least 75% of teachers report that students 
actively participate and engage in the classroom, the core condition is the 
presence of either high college expectations or high teacher influence, but 
neither condition is sufficient by itself. These core conditions have to be 
combined with other supports, including strong school leadership. When 
these conditions are met, schools show high levels of student classroom 
participation, regardless of the contextual characteristics of the school. 

Finally, to be a school with very high student participation, the core 
condition is selective enrollment, or the presence of a very high propor¬ 
tion of students entering high school with high achievement. Yet not all 
schools with a high proportion of students with strong academic prepara¬ 
tion achieve this outcome. Selective enrollment needs to be complement¬ 
ed by strong leadership and a wide array of organizational supports. This 
highlights the importance of principal and teacher leadership and orga¬ 
nizational supports over and above the prior achievement of the students. 
High schools face considerable challenges in organizing leadership and 
key supports to maintain a very high level of classroom participation even 
when most students come in adequately prepared. The challenge is even 
greater when the schools are not selective enrollment schools; there are 
no systematic solutions for such schools to achieve very high participation 
in classrooms in the data used for this study. 

With k conditions related to the outcome there are 2* possible combina¬ 
tions that can theoretically be observed. Therefore, the choice of condi¬ 
tions must be carefully made so that the solutions do not get needlessly 
complex. At the same time, it limits the ability to understand the relation¬ 
ship of all school organizational supports with important outcomes. In 
this study we chose one or two important constructs within each essential 
support (Bryk et al., 2010); ideally we would have liked to include a more 
comprehensive set of constructs. For example, the presence or absence of 
professional community was not found to be strongly related to the class¬ 
room participation outcomes that we examined. The particular aspect of 
professional community we examined was teacher peer collaboration and 
deprivatized practice. Other aspects of professional community such as 
collective responsibility, reflective dialogue, and teacher socialization were 
left out and could be studied in future research. 

The causal interpretations from the fuzzy set QCA results are limited by 
the cross-sectional nature of the data. Typically, QCA is used to uncover 
causal relationships of conditions to an outcome but they can be used to 
examine associative relationships as well (Ragin & Fiss, 2008). Although 
the sections discussing the methodology used terminology typically used 
in QCA literature denoting/implying causality, the results shown here only 
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describe associative relationships as the data is cross sectional. Another 
limitation is that the leadership and school organizational fuzzy sets were 
created from items from an annual teacher survey. Teachers may provide 
a biased view of school organization and classroom instruction. Requiring 
them to characterize things based on recalling events that stretched over 
an entire year may introduce additional errors. The implied causal direc¬ 
tion from leadership to organizational supports to instruction is adopted 
from the Bryk et al. (2010) conceptual framework (Figure 1) as a basis to 
interpret the QCA results; nothing inherent in the analysis itself implies 
this directionality. Unlike SEM where path diagrams can be specified to fit 
the conceptual framework, all conditions including leadership and other 
organizational factors have equal weight here as they relate to the outcome 
of classroom participation. The results suggest that leadership is a neces¬ 
sary condition for classroom participation but do not indicate whether 
there is a direct or indirect relationship. Perhaps the use of longitudinal 
data along with QCA methods can be used to uncover the directions of re¬ 
lationships among leadership, organizational, and instructional variables. 

The study also does not go beyond classroom instruction to actual stu¬ 
dent learning outcomes such as test scores and grade point averages. The 
difficulty involved with benchmarking school average test scores and GPA 
after controlling for prior achievement and demographic characteristics is 
a challenge. Examining student outcomes such as test scores and GPAs is 
the immediate and most compelling next step in this research. An impor¬ 
tant limitation of this study is the use of benchmarks that have not been 
validated or discussed extensively by education researchers. The use of 
benchmarks and calibration is relatively rare in the social sciences; educa¬ 
tion research is no exception. However, the primary goal of this study is 
not to derive the perfect benchmarks for classroom participation, school 
leadership, and other organizational factors. It is to propose the applica¬ 
bility of fuzzy set QCA as an additional tool along with correlation-based 
methods to study school leadership and organization. It is likely that the 
solutions will be different if the benchmarks used in this study are changed 
to recalibrate the fuzzy sets. The limited analysis we did with more con¬ 
servative benchmarks showed that the condition of selective enrollment 
emerged as the core condition for most outcomes. 7 Lastly, the potential 
of QCA to uncover relevant “fit among the important contextual, struc¬ 
tural, or strategic factors” (Doty & Click, 1994, p. 231) has not been fully 
exploited in this study as the conceptual framework was based on one 
model of school organization proposed by Bryk et al. (2010). Future stud¬ 
ies could explore different models of school leadership and organization, 
e.g., instructional and transformational leadership models to examine 
under what circumstance and through which organizational structures a 
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particular mode of leadership works, or if integrating both modes has dis¬ 
tinct advantages as found in prior research (Marks & Printy, 2003). 

In summary, the configurational approach using QCA methods proposed 
here offers an exciting direction for the study of school leadership and or¬ 
ganizational processes as they relate to instruction and student learning. It 
is important to note that researchers have cautioned against using QCA in 
a routinized manner and relying on software to produce the most parsimo¬ 
nious solutions rather than relying on theory (Cooper & Glaesser, 2011). 
Some researchers have also suggested that there is nothing genuinely new 
to the QCA methodology that is not already covered by regression analy¬ 
sis (Achen, 2005). Seawright (2005) for example, argues that QCA is not 
an improvement over regression methods, although it could be useful in 
highlighting the importance of higher order interactions and the value of 
considering nonlinear measurement models. In concluding, we maintain 
that this work is intended to begin a discussion of the usefulness of configu¬ 
ration approaches and QCA methods in school organizational research in 
addition to, rather than in opposition of, correlation-based methods. 
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Notes 

1. It would have been ideal to simultaneously examine multiple aspects of in¬ 
struction as the outcome but this is not possible in QCA analysis. Selecting another 
aspect of instruction such as academic demands or the quality of classroom discus¬ 
sions would again represent only one aspect of instruction. Moreover, procedures 
in QCA that require measures have substantively meaningful interpretations and 
benchmarks make it difficult to use a latent measure. 

2. There are multiple models that have been proposed to describe relation¬ 
ships between leadership, school organizational processes, student learning, and 
the influence of context (see for example, Bryk, et al., 2010; Knapp, Copland, & 
Talbert, 2003; Leithwood et al., 2004; Louis, Leithwood, et al., 2010). While there 
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are several unique features of each framework, there is also a great deal of overlap 
among them. 

3. Fiss (2011) explains easy and difficult counterfactuals in this way: 

Assume one has evidence that the combination of conditions A*B«~C (read: 
A and B but not C) leads to the presence of an outcome. No evidence exists as 
to whether the combination A»B»C (read: A and B and C) would lead to the 
outcome, but theoretical or substantive knowledge links the presence (not 
the absence) of C to the outcome. In such a situation, an easy counterfactual 
analysis indicates that both A»B»~C and A«B»C will lead to the outcome, 
and the expression can be reduced to A*B because whether C is absent or 
present has no effect on the outcome (p. 402). In contrast with reference 
to a difficult counterfactual, one might have evidence that the combination 
A«B«C leads to the outcome, but no evidence as to whether the combination 
A«B*~C also does so, This of course is the inverse of the situation above. . . . 
Theoretical or substantive knowledge links the presence, not absence of C to 
the outcome, and lacking an empirical instance of A«B«~C, it is much harder 
to determine whether C is in fact a redundant condition that can be dropped, 
thus simplifying the solution to merely A«B. (p. 403) 

4. Although the minimum consistency was set at 0.75, the final solutions after all 
analysis include configurations with consistencies below this value. 

5. Raw coverage indicates the share of the outcome explained by one the differ¬ 
ent solutions. 

6. Unique coverage indicates the share of the outcome explained by one of the 
solutions net of that solution’s overlap with other solutions identified. 

7. We also conducted analysis using more conservative benchmarks where 50% 
agreement by teachers was considered fully out in terms of membership in fuzzy 
sets. Selective enrollment schools along with leadership and almost all other or¬ 
ganizational supports was the only solution associated with high and very high 
student participation. 
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APPENDIX 

HLM Model Used to Create School-Level Estimates from Teacher Surveys 

To estimate school average scores for measures such as classroom par¬ 
ticipation, school leadership, and other organizational supports we used 
logistic hierarchical regression models with two levels: Level 1 represents 
teachers and Level 2 represents schools. The models were unconditional 
models because we did not adjust for teacher or school characteristics. 

Level-1 Model: 

Prob(OMfcOOTC^=l|Pj) = <p ij 
log[(pi/(l -<p</)] = r\ij 
r\ij= P Oj 

Level-2 Model: 

Po/= J00+ UOj 

Level-1 variance = l/[cpi/(l-cpg)], 

where the outcome is the survey measure (e.g., classroom participation, 
instructional leadership, etc.) that is estimated for teacher i in school j. At 
Level 1 the logistic regression model is written in terms of the log odds 
of the probability of a response denoted by (p,y. Here, tpy denotes the log 
odds of success (teacher endorsement) for a particular survey measure. 
At Level 2 we estimate the overall school average estimate y oo of the log 
odds of success. HLM provides Empirical Bayes estimates for calculating 
individual school estimates for the outcome. These can be converted to 
probabilities to yield a school specific estimate for the probability of en¬ 
dorsement on teacher survey outcome. 
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